In recent years, MR systems are known that allow an observer to experience mixed reality (MR) in which a virtual object seems to exist in a physical space. In an MR system, a composite image is generated by aligning and combining a real image of a physical space that is captured from the observer's observation point and a computer graphics (CG) image that represents a virtual object. The composite image is then represented to the observer.
At this time, a head mounted display device (HMD) is generally utilized as a display apparatus that represents the composite image to the observer. Normally, an image-sensing device for capturing a physical space image of the observer's observation point is provided in a HMD that is used with an MR system. In a case in which the HMD is configured so as to be capable of representing independent video images in the left and right eyes, respectively, in many cases the image-sensing device will also have independent configurations for the right eye and the left eye. Hereunder, a HMD provided with an image-sensing device is referred to as a “head mounted image-sensing display device”.
In this kind of head mounted image-sensing display device, when the image sensing directions of a pair of image-sensing devices and the parallax of a pair of CG images that are combined with left and right images that are picked up by the image-sensing device do not match, the CG images that are observed appear unnatural and seamlessness with the physical space is not obtained. There is also a risk that the observer will feel a sense of fatigue while observing the unnatural image.
To solve this problem, technology has been proposed that adjusts a display unit to conform to the eye width of the observer and generates a CG image including parallax in accordance with the adjustment amount thereof. This technology also attempts to substantially match the optical axis of an image-sensing optical system that corresponds to both the left and right eyes and the optical axis of a display optical system (Japanese Patent Laid-Open No. 2006-108868).
According to the aforementioned technology, by performing eye width adjustment, generating an image by taking into account the parallax thereof, and making the image sensing optical axis and the display optical axis match, it seems that an effect can be expected whereby the parallax of the virtual image and the actual image are matched. However, this effect can only be expected in a case in which, for a head mounted image-sensing display device, assembly is carried out so that the values for a base line length and an image sensing direction of an image-sensing device are in accordance with the design values.
When assembly is performed with values that differ from the design values, ultimately a mismatch between the parallax of a real image and a CG image is not overcome. Therefore, there is still a risk that the observer wearing the head mounted image-sensing display device will feel a sense of discomfort and a sense of fatigue.